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DETAILED ACTION 

Continued Examination Under 37 CFR LI 14 

1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on December 7, 2004, has been entered. 

Response to Amendment 

2. All of the previous claims (1-39) have been cancelled. A rejection of new claims 40-69 
follows. 

Claim Objections 

3. Claim 64 is objected to as failing to comply with 37 CFR 1 .75(a) for failing to 
particularly point out and distinctly claim the subject matter which applicant regards as the 
invention. 

Claim 64 recites the limitation "the image output apparatus" on line 5. There is 
insufficient antecedent basis for this limitation in the claim. For examination purposes, the claim 
will be treated as if it reads, "an image output apparatus 5 '. 
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Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

6. Claims 40-50, 58, 59, and 63-69 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ooyama (Japanese Patent Application Publication No. 09-172649) in view of 
Dandliker (U.S. Patent No. 3,922,093). 

Regarding claims 40 and 67, Ooyama discloses an image processing apparatus 
(processor 30 in Drawing 1) for processing an image of an object taken by an image input 
apparatus (spectrum picture photography means 10), said image input apparatus comprising: 
a reproducing environment converting unit (spectrum conversion means 
30b) which combines a plurality of images of the object (page 4, lines 25-30, of 
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the previously provided machine translation) which are taken by the image input 
apparatus under a plurality of respective environments to convert the image of the 
object into an observation image according to an environment in which the 
observation image of the object is to be observed (see page 4, lines 40-43), based 
on: (i) photographing environment information acquired at a photographing time 
at which the object is photographed with the image input apparatus (light 
spectrum distribution means 20 on the photographing side of the system measures 
lighting data; see page 4, lines 25-30), and (ii) observing environment information 
acquired at an observing time at which the observation image of the object is 
observed (the system also detects the lighting spectrum on the reproduction side 
of the system; see page 5, lines 33-34). 
Ooyama is silent with regard to detecting object feel-of-material information. 
Dandliker discloses a system for measuring the roughness of a surface and captures: 
object feel-of-material information regarding texture of the object (the 
number N is calculated, which is a direct measure of the roughness of the paper; 
see column 4, lines 3-8), said object feel-of-material information being acquired 
from the plurality of images of the object (see column 4, lines 32-34). 
An advantage of measuring the feel of a surface is that a system can determine whether a 
bright spot on an object is the result of glossiness or an actual lightly colored area. For this 
reason, it would have been obvious at the time of invention to have Ooyama 5 s system determine 
feel-of-material information about a subject. 

Regarding claim 41, Dandliker teaches: 
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the object feel-of-material information includes light reflection 
information regarding reflection of light from the object (see column 3, lines 66- 
68). 

Regarding claim 42, Dandliker teaches: 

the light reflection information includes specular reflection information 
(see column 3, line 68, through column 4, line 2) regarding specular reflection at a 
surface of the object, said specular reflection varying in accordance with an angle 
of the light with respect to the surface of the object (the specular reflection is the 
result of glancing angle a; see column 2, lines 48-50). 

Regarding claim 43, Dandliker teaches: 

the light reflection information further includes diffuse reflection 
information regarding diffuse reflection at the surface of the object (as shown in 
Figure 4, TV camera 15 captures scattered light 26 in addition to reflected light 
27; see column 3, line 64, through column 4, line 2). 

Regarding claim 44, Dandliker teaches: 

the light reflection information is acquired based on a geometrical 
relationship (see the angles of specular reflection a in Figure 1 A; the operation of 
the embodiment in Figure 3 is the same as the embodiment in Figure 1 A, as noted 
in column 3, line 50) between the image input apparatus (which receives the beam 
1 in Figure 1 A), the object (surface 8) and a light source (21) for illuminating the 
object at the photographing time (the measured light intensity I shown in diagram 
23 is the result of light shining at angle a; see column 2, lines 51-66), and wherein 
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the geometrical relationship is determined based on the plurality of images (the 
light beam moves over time in order to produce diagram 23; see column 2, lines 
60-66). 

Regarding claim 45, Ooyama teaches: 

the observing environment information includes light color information 
regarding a color of illuminating light emitted onto a place where the observation 
image of the object is observed at the observing time (see page 5, lines 33-35). 
Regarding claim 46, Ooyama teaches: 

the photographing environment information includes light color 
information regarding a color of illuminating light emitted onto the object at the 
photographing time (see page 5, lines 27-32). 
Regarding claim 47, Ooyama teaches: 

the photographing environment information includes light spectrum 
information regarding a spectrum of illuminating light emitted onto the object at 
the photographing time (see page 5, lines 27-32); 

the observing environment information includes light spectrum 
information regarding a spectrum of illuminating light emitted onto a place where 
the observation image of the object is observed at the observing time (see page 5, 
lines 33-35); and 

the reproducing environment converting unit converts the image of the 
object based on a difference between the spectrum of the illuminating light 
emitted onto the object at the photographing time and the spectrum of the 
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illuminating light emitted onto the place where the observation image of the 
object is observed at the observing time (see page 5, lines 36-41). 
Regarding claim 48, Ooyama teaches: 

the photographing environment information includes light spectrum 
information regarding a spectrum of the illuminating light (see page 5, lines 27- 
32). 

Regarding claim 49, Ooyama teaches: 

the photographing environment information includes at least one of: (i) 
object form information regarding a form of the object, (ii) object color 
information regarding a color of the object (image data of the subject is captured 
in color to be reproduced in color; see page 6, lines 44-46), (iii) object direction 
information regarding a direction of the object with respect to the image input 
apparatus, and (iv) object position information regarding a position of the object 
with respect to the image input apparatus. 
Regarding claim 50, Ooyama teaches: 

the observing environment information includes light spectrum 
information regarding a spectrum of the illuminating light (see page 5, lines 33- 
35). 

Regarding claim 58, Ooyama teaches that the image is captured by a single multi- 
spectrum camera (see page 6, lines 18-19). 

Regarding claim 59, Ooyama teaches: 
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the observing environment information is information regarding an 
environment in which the observation image of the object output from the image 
output apparatus is observed (see page 5, lines 33-35). 
Regarding claim 63, Ooyama teaches: 

the reproducing environment converting unit comprises an image 
interpolating and composing unit (interpolator 32 and spectrum memory 33) for 
combining the plurality of images (interpolator 32 outputs data to spectrum 
memory 33, which collects the data; see page 7, lines 37-39). 
Regarding claim 64, Ooyama teaches: 

the photographing environment information includes information 
regarding the image input apparatus, and the observing environment information 
includes information regarding the image output apparatus (image processing 
occurs by converting the color spaces used by the camera to the color space used 
by the display; see page 13, lines 7-17). 
Regarding claims 65 and 68, Ooyama discloses an image processing apparatus 
(processor 30 in Drawing 1) for processing an image of an object taken by an image input 
apparatus (spectrum picture photography means 10), said image processing apparatus 
comprising: 

a reproducing environment variable image data producing unit (spectrum 
conversion means 30b) which combines a plurality of images of the object (page 
4, lines 25-30, of the previously provided machine translation) taken under a 
respective plurality of environments to produce reproducing environment variable 
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image data based on photographing environment information acquired at a 
photographing time at which the object is photographed by the image input 
apparatus (light spectrum distribution means 20 on the photographing side of the 
system measures lighting data; see page 4, lines 25-30). 
Ooyama is silent with regard to detecting object feel-of-material information. 
Dandliker discloses a system for measuring the roughness of a surface and captures: 

object feel-of-material information regarding a texture of the object (the 
number N is calculated, which is a direct measure of the roughness of the paper; 
see column 4, lines 3-8), said object feel-of-material information being acquired 
from the plurality of images of the object (see column 4, lines 32-34). 
An advantage of measuring the feel of a surface is that a system can determine whether a 
bright spot on an object is the result of glossiness or an actual lightly colored area. For this 
reason, it would have been obvious at the time of invention to have Ooyama' s system determine 
feel-of-material information about a subject. 

Regarding claims 66 and 69, Ooyama discloses: 

a reproducing environment converting unit (spectrum conversion means 
30b) which converts the reproducing environment variable image data into an 
observation image in accordance with an environment in which the observation 
image of the object is observed (see page 4, lines 40-43), by applying observing 
environment information acquired at an observing time at which the observation 
image is to the reproducing environment variable image data (see page 5, lines 
33-35). 
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7. Claims 51 and 52 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ooyama in view of Dandliker and further in view of Minami (U.S. Patent No. 6,014,472). 

Claim 51 may be treated like claim 50. However, Ooyama and Dandliker are both silent 
with regard to including at least one of an observation illuminating light position converting unit 
and an observation illuminating light form converting unit. 

Minami discloses an image processing device, including: 

an observation illuminating light position converting unit (shadow signal 
producing section 20) which converts the plurality of images (a video signal is 
processed; see column 4, lines 21-22) into images which are adapted to be 
respectively acquired when the at least one light source for illuminating the object 
in the image is moved to an arbitrary position at the observing time (the operator 
selects a position for a light source, resulting in the generation of an appropriate 
shadow; see column 30, line 46, through column 31, line 28). 
An advantage of producing a video signal with an object relocated to a desired position 
with respect to a light source is that the position of the object may be improved without the 
expense of re-shooting the scene. For this reason, it would have been obvious at the time of 
invention to have Ooyama' s system include the virtual light positioning system described by 
Minami. 

Claim 52 may be treated like claim 40. However, Ooyama and Dandliker are both silent 
with regard to the observing environment information including at least one of object position 
information, object direction information, and observer position information. 
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Minami discloses an image processing device for setting observing environment 
parameters, including: 

object position information (the operator selects a position for a subject, 
resulting in the generation of an appropriate shadow; see column 30, line 46, 
through column 31, line 28) regarding a position of the object in the observation 
image. 

An advantage of producing a video signal with an operator-specified position a subject is 
that the position of the subject may be improved without the expense of re-shooting the scene. 
For this reason, it would have been obvious at the time of invention to have Ooyama's system 
include the virtual light positioning system described by Minami. 

8. Claim 53 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ooyama in view 
of Dandliker and Minami and further in view of Katayama (U.S. Patent No. 6,256,035). 

Claim 53 may be treated like claim 52. However, Minami is silent with regard to 
including at least one of an object moving unit, an object rotating unit, and an observation 
position converting unit. 

Katayama discloses an imaging system that includes: 

an object moving unit (CPU 102) which converts the plurality of images 
(images are captured by camera 803) into images which are adapted to be 
respectively acquired when the object in the image is directed in an arbitrary 
direction, based on the object position information (the user may enter a point at 
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which to place the "virtual camera,", thus changing the viewer's observation 

direction of the object; see column 5, lines 16-35). 
An advantage of producing a video signal of an object relocated at a desired position is 
that the viewer may obtain information upon demand about an object. A viewer can therefore 
gain a more accurate impression of the object. For this reason, it would have been obvious at the 
time of invention to have Ooyama's system include the virtual camera positioning system 
described by Katayama. 

9. Claims 54 and 55 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ooyama in view of Dandliker and further in view of Cabral (U.S. Patent No. 6,697,062). 
Claim 54 may be treated like claim 40. Additionally, Ooyama discloses: 

a reproduction environment variable image data producing unit 
(computing element 35) which produces reproduction environment variable image 
data from the image of the object taken by the image input apparatus by using the 
photographing environment information (calculations on the image data are 
performed based on the lighting detected on the photographing side of the system; 
see page 9, lines 22-28); and 

a data transmitting apparatus (communication interface equipment 40 and 
45 in Drawing 3) which transmits the reproduction environment variable image 
data to the reproducing environment converting unit through one of a portable 
recording medium and a network (via the communication line shown in Drawing 
3). 
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Ooyama is silent with regard to producing reproduction environment variable image data 
from object feel-of-material information. 

Cabral discloses an image rendering system that uses a map of an object including the 
object's texture to produce a three-dimensional reproduction of the object (see column 6, lines 
58-62, and column 7, lines 50-55). 

An advantage to rendering an image using stored feel-of-material information is that a 
more realistic image may be produced when the image's lighting is varied. For this reason, it 
would have been obvious at the time of invention to have Ooyama' s system use object feel-of- 
material information in performing a reproduction. 
Regarding claim 55, Ooyama discloses: 

the reproducing environment converting unit converts the transmitted 
reproduction environment variable image data into the observation image (see 
page 9, lines 22-28). 

10. Claims 56 and 57 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ooyama in view of Dandliker and further in view of Jones (U.S. Patent No. 3,564,988). 

Claim 56 may be treated like claim 40. However, Ooyama and Dandliker are both silent 
with regard to including a turntable. 

Jones discloses a device for three-dimensional photography (see Figure 1), wherein: 

the image input apparatus controls a turntable for rotating the object at a 
desired angle (turntable 60 is rotated under control of the camera; see column 1 , 
line 74, through column 2, line 7), and picks up the plurality of images of the 
object at a plurality of angles (see column 3, lines 8-13). 
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As stated in column 1, lines 54-59, an advantage to using a turntable is that the subject 
can be captured at a variety of angles. For this reason, it would have been obvious at the time of 
invention to have Ooyama' s system include a turntable, such as the one described by Dandliker. 

Regarding claim 57, Jones teaches that turntable 60 may be both rotated and tilted 
relative to camera assembly 10 (see column 3, lines 44-45). 

1 1 . Claims 60 and 61 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ooyama in view of Dandliker and further in view of Ishibashi (U.S. Patent No. 6,215,461). 

Claim 60 may be treated like claim 59. However, Ooyama and Dandliker are both silent 
with regard to including a head-mounted display. 
As shown in Figure 1, Ishibashi discloses: 

a head mounted display (2) which is adapted to display at least one of a 
stereoscopic observation image of the object (see column 3, lines 14-24) and a 
holographic observation image of the object. 
An advantage of using a head-mounted display is that a more realistic image can be 
provided to a viewer. For this reason, it would have been obvious at the time of invention to 
have Ooyama's system include a head-mounted display, such as the one described by Ishibashi. 
Regarding claim 61, Ishibashi teaches: 

the head mounted display comprises a gyroscopic sensor (405 and 406) 
which detects a change of the observing environment information (see column 4, 
lines 3-5), and which changes the displayed observation image in accordance with 
movement of the head mounted display (the gyroscopes detect the movement of 
the user's head and move camera 1 accordingly; see column 4, lines 48-54). 
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12. Claim 62 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ooyama in view 
of Dandliker and further in view of Wright (U.S. Patent No 4,757,379). 

Claim 62 may be treated like claim 40. Additionally, Ooyama teaches that the image 
input device is programmed to rotate filter 2 while capturing a plurality of images (see page 7, 
lines 27-43). However, Ooyama is silent with regard to illuminating the plurality of images at 
different angles by an arbitrarily movable illumination source. 

Wright discloses an apparatus and method for acquisition of 3D images (see Figure 4), 
including: 

wherein when the plurality of images are picked up (two image scans 
occur; see column 5, lines 36-38), the object is illuminated at different angles by a 
source of illumination (light sources LI and L2 are used in successive scans; see 
column 5, lines 40-42) for illuminating the object on a photographing side, said 
source of illumination being arbitrarily movable (see column 5, lines 38-40). 
An advantage of moving an illumination source to an arbitrary point is that shadows may 
be reduced in complex objects. For this reason, it would have been obvious at the time of 
invention to have Ooyama' s system include an arbitrarily movable illumination source. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason Whipkey, whose telephone number is (703) 305-1819 or 
(571) 272-7321 beginning March 1 5 2005. The examiner can normally be reached Monday 
through Friday from 8:30 A.M. to 6:00 P.M. eastern standard time, alternating Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wendy Garber, can be reached at (703) 305-4929. The fax phone number for the 
organization where this application is assigned is (703) 872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



JTW 

February 18, 2005 




